Motion and disparity field estimation using rate-distortion optimization

A rate-distortion framework is used to de ne a displacement vectoreld estimation technique for use in video coding. This technique achieves maximum reconstructed image quality under the constraint of a target bitrate for the coding of the vector sequence. The technique may be adapted so as to limit its smoothing e ect to homogeneous areas and avoid highly textured areas and edges. Use of this technique is evaluated for two application areas in which the need for high compression of displacement vector elds is particularly acute. The rst is motioneld coding for very low bit rate image sequence transmission as in videophone applications. The second application area is coding for the transmission of dense disparity elds. This is needed for the generation at the receiver of intermediate viewpoints through spatial interpolation. It is also needed in a number of other applications requiring accurate depth knowledge, including 3D medical data transmission and transmission of scenes to be postprocessed using depth-keyed segmentation. Experimental results illustrating the performance of the proposed technique in these application areas are presented and evaluated. Subject terms: Very low bit-rate coding; rate-distortion theory; vector eld coding; depth map coding. This work was supported by the EU CEC Project PANORAMA (ACTS project 092) and the COST 211ter project.